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(57) ABSTRACT 

The invention relates to a method and a device for the online 
processing of mail items to be forwarded. In the process, an 
image of the mail item is taken and the mail item is stored 
in an intermediate storage area. Following digitizing, the 
areas with interesting information are determined and for- 
warding instructions, names and addresses of recipients and, 
in the case of a return notation by a delivery person, the 
sender address are read automatically. The results are 
checked against a name address data bank and, if they do not 
coincide, are checked against a forwarding directory. Items 
that are not read clearly automatically, but for which at least 
one forwarding instruction exists for possible delivery 
locations, are evaluated with priority in a multistage video 
coding process by taking into account the printed-on instruc- 
tions for forwarding and return. In that case, if the first video 
coding is incomplete, the items are again read automatically 
by taking into account the coding results and, if necessary, 
an additional video coding. The items leaving the interme- 
diate storage area are then imprinted with forwarding 
instructions and distributed correspondingly. 

9 Claims, 13 Drawing Sheets 
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Sender 
Sender Street 
33333 Sender City 



Recipient 
old street 145 
12345 old city 




New address, RED, FAT PRINT 

- printed online with second printer in the OCR 

- printed directly on the item 

■ printed beside the recipient's address (left or right) 

- with or without keyword FORWARDING in line 1 
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do not forward! 



Recipient 
old street 145 
12345 old city 




RETURN TO SENDER 
"Change of address" 
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barcode for sender address 



return notice, RED, FAT PRINT 

• printed online with second printer in the OCR 

- printed directly on the item 

- printed beside the recipient's address (left or right) 

- including reason for return 
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return notice, RED, FAT PRINT 

- printed online with second printer in the OCR 

• printed directly on the item 

• printed beside the recipient's address (left or right) 

- including reason for return 
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Sender 
Sender Street 
33333 Sender City 



Herr Empfanger 
Alte StraBe 145 
12345 Alte Stadt 




ID code/barcode of sender address & 

optional return notice, RED, FAT PRINT: 

■ printed online with second printer in the OCR 

- printed directly on the item 

- printed over the barcode 
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METHOD AND DEVICE FOR ONLINE address as well as a list of names of persons who have placed 

PROCESSING OF FORWARDING MAIL forwarding requests. The service personnel then search the 

list for the name on the envelope. As soon as this has taken 

BACKGROUND OF THE INVENTION place, a new label with a new address is applied either 

5 automatically or manually. The label in this case is posi- 

The invention relates to an online processing of mail to be tioQcd such tha , the oW barcodc ^ ^vered. As a result, the 

forwarded and methods for improving the optical code letter can once again be read and coded automatically, along 

recognition of postal addresses through use of video encod- ^ ^ remaimilg itcms> disadvantage of this method 

ing techniques. ^ me n i gh manua l expenditure for the processing and the 

The automatic reading of mailing addresses with the aid 10 f ac t that items to be forwarded must be separated from the 

of optical code recognition (OCR) is a technology repre- normal letter flow. This results in delays and detours for 

senting a fixture in everyday postal life. Processing rates of items to be forwarded. (Computer Forwarding System II of 

up to 10 items per second (36,000 items per hour) are the USPS). 

possible with modem address readers. The reliability of the More a d va nced systems use the OCR technology. In that 

recognition depends to a large degree on the character design 15 case> items t0 ^ f orwar d e d are again intercepted by the 

and the quality of the print. If the recognition is successful, delivery person, are provided with stamps such as "moved, 

a machine-readable barcode is printed on. This barcode address unknown/' and are sent to an automatic forwarding 

permits the step-by-step mechanical separation of the items ccntcr ^ f orwar d mg center is provided with address 

until the desired sorting has been achieved. The bar coding readerS7 wn i c h automatically detect the address of the sender 

technology in particular has made possible the step-by-step 20 an d that of the recipient for all items to be forwarded. The 

sorting (multi-pass) of the items until they are sorted in the new address ^ determined with the aid of a forwarding data 

delivery sequence of the delivery person. bank a ddress reader can read and evaluate all com- 

However, since the reading rates are so different, the ments by the delivery person, advance instructions and other 

automatic operation must be supported by various manual features. The evaluation of the features makes it possible to 

interventions. The simplest method of processing items that 25 decide on what to do with the items. In another automation 

cannot be read is to have them rejected in the machine and s t e p, a label with the new address is affixed to the mail. After 

to sort them by hand, using sorting cabinets. However, the the barcode has been printed on as well, the mail is again 

costs for this type of sorting and delivery are unreasonably inserted into the regular flow of letters for delivery, 

high, especially with increasing personnel costs. Added to ^ jj s Pat No 5^22,821 discloses a system for 

this is the fact that hand-sorted items cannot be distributed 30 f orwar ding incorrectly addressed items, which detects items 

automatically in any phase. Thus, two letter flows develop, tQ be forwarde d in an earlier stage of the letter processing 

which must be combined once more at some points. and differentiates between items that can be read by a 

In order to avoid the disadvantages of a manual sorting of machine and those that cannot be read by a machine, 

items rejected in the OCR, various approaches to the manual After uking a p i cture 0 f the address side of the mail, 

coding of items were developed. All methods are based on 35 mc ] ud i Dg t he digitizing, the name and address of the recipi- 

service personnel typing in parts of the address, so that the ent m read by means 0 f an OCR reader. Following that, the 

target of "providing the mail with a barcode" is reached. rade 0 f ^ distribution location is determined in an 

Once the items are provided with a barcode, they can be \js?S ZIP+4 database. Subsequently, it is determined in an 

processed in the same way as items that are provided directly usps Nat j ona i change of Address (NCOA) database 

with a barcode during the OCR. 40 whether any forwarding requests have been entered. At the 

The forwarding of items is a problem that has been known same ti me> written forwarding instructions on the items are 

for a long time. Still, the majority of items nowadays are evaluated automatically. During the time required for deter- 

forwarded manually. mining the correct address, the items are located in a 

Nowadays, items with the incorrect (old) address that are 45 mechanical delay loop/intermediate storage. If the address, 

to be manually forwarded are sent to the delivery person at including the ZIP+4 Code cannot be determined automati- 

the old delivery area. For the delivery sequence sorting of cally during the time the respective mail spends in the delay 

the items, the delivery person uses a distribution cabinet, for loop, the images are stored with an identification number, 

which the number of compartments equals the maximum This ID Number is subsequently printed onto the mail as a 

number of delivery points in the delivery area of the delivery 5Q barcode and the further processing occurs off-line with the 

person. If a forwarding request is made, a forwarding aid of video coding in the above-described manner, wherein 

reminder card with the new address is sent to the delivery these items must be removed from the flow of items and 

person in the old delivery area. collected in a special sorting compartment. If the correct 

The forwarding reminder card is deposited in the respec- address is determined during the time the respective mail 

tive compartment. During the delivery sequence sorting, 55 s P ends m th / dela y loo P'. a labc * * ap ? Ued T a1 

forwarding mail in compartments with forwarding reminder incorrect address, onto which the new address is printed. A 

cards are not sorted in. Instead, the delivery person writes barcode sorter subsequenUy distributes the items on the 

the new address onto the envelope and hands the mail over basis of the pnnted-on new barcode. This operation takes 

to the manual-processing location. P lace on hne< 

In systems that have been modified, the delivery person 60 SUMMARY OF THE INVENTION 
identifies letters that must be forwarded, removes these 

letters from the flow of letters and sends them to a central The present invention is based on the problem of incor- 

location for processing letters to be forwarded. With the aid porating the video coding technique into the OCR reading 

of extraction techniques, service personnel will input the technique to increase the number of items that can be 

incorrect address into a computer that is connected to a 65 processed on line. This should occur in such a way that 

central data bank. The data bank contains information on all during the time the items spend in a delay loop, additional 

forwarding requests and displays on the screen the new items that cannot be read by means of the OCR reading 
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technique can be evaluated. Whether and to what location a 
mail item must be forwarded follows from the forwarding 
inscription or the name-related and delivery-location related 
forwarding directories. 

The advantages achieved with the invention in particular 
are that names and addresses of recipients, which cannot be 
recognized automatically by means of the OCR reader, can 
be incorporated into the online processing as a result of the 
prioritized video coding according to the invention of the 
items to be forwarded. Further advantageous embodiments 
of the invention follow from the dependent claims. 

The multistage incorporating of the video coding into the 
automatic OCR reading process, permits a quick recognition 
of names and addresses with an effective use of the video 
coding personnel. This process becomes particularly advan- 
tageous through the effective use of extraction coding and 
the selection coding, which makes it easier to reach deci- 
sions for the video coding personnel. 

In order to improve the reading rate, other advantageous 
embodiments according the invention provide that even in 
cases where the automatic reading of forwarding and return 
instructions or sender addresses is not very clear, the items 
are sent to a video coding. 

BRIEF DESCRIPTION OF THE DRAWINGS 



30 



40 



In the following, the invention is described in further 
detail with the aid of the drawings. 
Shown are in: 

FIG. 1 An operational block diagram for the online 
processing system; 

FIG. 2 A flow diagram of the operational sequence of the 
online processing systems (part 1); 

FIG. 3 A flow diagram of the operational sequence (part 35 

2); 

FIG. 4 A flow diagram of the operational sequence of the 
online processing system (part 3); 

FIG. 5 A flow diagram of the operational sequence of the 
online processing system (part 4) for the video coding; 

FIG. 6 An overview of the data flow during the multistage 
video coding (phase 3); 

FIG. 7 A letter to be forwarded; 

FIG. 8 A letter to be returned to the sender, 

FIG. 9 A letter to be returned to the sender where the 
sender has printed his/her postal customer number and 
advance instruction encoded into the address field for the 
recipient; 

FIG. 10 A letter, which was not recognized as letter to be 
forwarded when the mail arrived; 

FIG. 11 A letter according to FIG. 10, stamped for return 
by the delivery person, with an empty label above the 
barcode; 

FIG. 12 The letter according to FIG. 11, following treat- 
ment in the system according to the invention; 

FIG. 13 A block diagram of the machine configuration for 
the online system. 
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code recognition" (OCR) and "representation at a video 
coding station." Following that, the scanner sends the image 
301 to the address block determination (address block 
location (ABL)) 207, a detector for advance instructions 213 
and a detector for delivery instructions 218. The mail itself 
is buffered physically in a delay segment 202 of the address 
reader. 

The address block locator (ABL) 207 determines the 
position and dimensions of the address block. The pixel 
image of the discovered address block is transmitted to an 
optical code recognition 209, which converts the pixel 
image into a string of codes with credibility evaluations and 
alternatives for the individual codes. All text parts of the 
address, which are necessary for the determination of a 
delivery location, are detected. A recognition system control 
215 transmits the recognition results 309 to a first compara- 
tor 216, which improves the recognition results by using 
contextual knowledge 307 of all possible addresses from a 
name address data bank, also called a dictionary. If a 
successful comparison with the dictionary is not possible, 
the address block is forwarded to the video coding. 

The code recognition system is supplemented by an 
online video coding system with 3 different phases: 
Phase 1: video coding, first stage 210: 

The video coding operator inputs a short, easily rec- 
ognizable extract with fixed length, which resolves 
the departure information and a portion of the arrival 
information. If an unambiguous dictionary compari- 
son is possible with this extract, then the address is 
resolved up to the delivery location. 
Phase 2: OCR repeat attempt 211: 

For addresses where the extract allows more than one 
dictionary entry as correct resolution, the code rec- 
ognition is started again with the additional knowl- 
edge obtained in the video coding stage 1. This 
additional information decisively increases the prob- 
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DETAILED DESCRIPTION OF THE 
INVENTION 

According to FIG. 1, the reference number 200 designates 
mail, e.g. an envelope, which is checked to determine 
whether it must be forwarded or sent back. A gray-key 
scanner 201 optically scans the envelope surface and pre- 
pares the scanned image for the subsequent steps of "optical 



60 



65 



ability of a correct OCR decision during the 2 n< 
attempt. 

Phase 3: video coding stage 2 (also called selection phase) 
212: 

The addresses, which are not clearly resolved following 
the repeat OCR attempt, are locked onto a video 
coding station together with the possible, correct 
dictionary entries. The video coding personnel is 
urged to make the final decision by depressing a key. 
A recognition system control 215 monitors and controls 
all necessary interactions between the optical code recogni- 
tion and the video coding system. 

After passing through the optical code recognition and 
possibly also through one or several phases of the video 
coding system and following a dictionary comparison, the 
delivery location for the mail to be checked is known. As 
soon as the delivery location has been found, the signal 315 
is transmitted to a second comparator 217. As a result of 
signal 308 from a forwarding directory, the second com- 
parator has aD the information with respect to the forwarding 
requests (respectively with the new and old address). If it 
detects a forwarding candidate in the mail to be checked, 
then a signal 306 is transmitted to an address change system 
214. 

Other signals are also transmitted to the address change 
system: A delivery instruction 218 detection system detects 
the imprints of the return stamps, used by the delivery 
persons and transmits a signal 311 if such a stamp is 
recognized. A device for detecting advance instructions 
detects such advance instructions as "please return to sender 
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if recipient has moved," and sends a signal 310 to the 
address change system 214 if an advance instruction is 
detected. For a return of the mail, it is also necessary to read 
the sender address. This task is taken over by the optical 
code recognition and the video coding system. 

The address change system 214 determines the target 
address for the mail to be checked on the basis of the signals 
306, 310 and 311. If the mail is to be forwarded, then the 
target address is the new address of the recipient. For a 
return, the address of the sender is the target address for the 
mail. The target address 303 barcode is affixed with a 
barcode printer 203 to the mail. In addition, a forwarding 
instruction and the new address, or a return instruction are 
printed with a text printer 204 in clear text onto the mail. A 
position determination 208 determines the position of the 
instructions and additionally of the new address only if the 
mail is to be forwarded. The position determination is 
provided by the address block determination 207 with 
information on the length, width and positioning of the 
address block on the mail (signal 300) as well as the 
dimensions for the mail itself 302. It transmits a signal 304 
to the text printer, which designates the position of the text 
to be printed on. Normally, the text is printed to the right or 
left of the (old) receiver address. In order to emphasize this 
text, it is printed in bold letters and the color red. The 
delivery person should be able to read the printed on text 
even if the mail has a background pattern at this location. 

Subsequently, the mail is sorted according to its target 
address 205 and is transported to the respective rough 
distribution compartment 206. 

In general, items that do not need to be diverted 
(forwarding or return) are only imprinted with the target 
barcodes. 

All processes described so far are carried out while the 
mail is located in the delay loop of an address reader. For 
that reason, a significant number of items can be processed 
online and, if necessary, also diverted. 

The related FIGS. 2, 3 and 4 provide a detailed overview 
of the processing sequence for the automatic processing of 
items to be forwarded and returned. A mail item 100 is 
scanned during the processing step 101. The scanned image 
is transmitted to the device for detecting advance instruc- 
tions 102b and to the device for detecting delivery instruc- 
tions 102c. If a delivery instruction is detected, then a 
processing step for the return is implemented. If no delivery 
instruction exists, the pixel image is transmitted to the image 
recognition 102a. The complete address information, 
including the delivery location and receiver name, are 
detected therein. 

In order to improve the reading result, context informa- 
tion is used for the subsequent dictionary comparison 103. 
The dictionary contains all address information up to the 
individual delivery locations. If the comparison with the 
dictionary 104 does not provide a clear result, the missing 
information is supplemented with a 3-phase video coding 
105. With the aid of the delivery location, determined in this 
way, it is determined in a forwarding data bank 106 whether 
at least one forwarding inquiry 107 exists at this delivery 
location. All possible, correct and different spellings of the 
addresses are stored in the dictionary as well as in the 
forwarding data bank, so as to ensure the highest possible 
recognition rate. 

If the delivery location to be examined does not contain 
a forwarding request, which is the case in about 95% of the 
time, the target address is affixed in a step 108 in the form 
of a barcode onto the mail, and the mail is sorted according 
to its target address 109. 
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In the other case, that is if the delivery location has at least 
one forwarding request, the name of the recipient must be 
identified. For that purpose, the forwarding data bank con- 
tains a complete list of the persons associated with this 

5 delivery location. The name read by the code recognition 
system is compared in step 115 to the list of names from the 
forwarding data bank. Once the (recognized) name of the 
recipient is found in the forwarding list 116, possible 
advance instructions are considered during the subsequent 
step. The recognition of these advance instructions W2b 
takes place parallel to other processes, such as code recog- 
nition and access to the dictionary. However, the results of 
this recognition are evaluated only if a mail item was 
recognized as mail to be forwarded. The recognition of the 
advance instructions is important, particularly since in many 

15 countries the majority of items are not forwarded if there is 
a change of address, but must be returned to the sender 
(especially large-scale senders) since the senders want to 
take care of their address data banks. 
The first advance instruction that is checked is "return to 

20 sender." For this, the recognized text is compared 120a to a 
list of permissible advance instruction texts. If the mail must 
be returned, it is treated as shown in FIG. 4. If the mail must 
be forwarded, it is treated as shown in FIG. 3. 

FIG. 3 

Forwarding of items: 

If the mail is provided with an advance instruction of the 
type that has "do not forward — destroy the mail" printed on 
it (e.g. for information mail in Germany), this is detected by 
comparing the recognized advance instruction text to a list 
of permitted texts Yltob. In that case, the letter is sent to a 
manual processing 121. 

If no advance instruction was detected, the mail is for- 
warded. For this, the barcode of the new recipient address is 
35 printed on as target code 150 and an instruction to forward 
the mail 151, as well as the new address 152, are printed on 
as clear text. The mail is then sorted in step 153 according 
to the new address. 

FIG, 4 

40 Returning the mail: 

If a mail item has been identified as return mail through 
recognition of an advance instruction or a delivery notation, 
the sender address must be read in step 130a. A dictionary 
comparison 131a takes place after that. If no clear recogni- 

45 tion result for the sender address 132a exists, the recognition 
is supplemented through video coding 133a. As soon as the 
sender address has been identified successfully, it is affixed 
in step 131 to the letter in the form of a barcode as the target 
code. Furthermore, a return notation ana* — if known — the 

50 reason for returning the mail are printed in clear text 132 
onto the mail. The mail is subsequently sorted in step 133 
according to the sender address. 

In addition, the sender address and the new, as well as the 
old recipient address 140 of each return mail are entered into 

55 the data bank. Address change reports are regularly com- 
piled in step 141 and sent to the customer if the customer 
(sender) desires this in decision 134, thus allowing the 
sender to cultivate his/her address list. 

The FIGS. 5 and 6 again provide a detailed overview over 

60 the 3-phase video coding previously mentioned in connec- 
tion with FIG. 1: 

Rejected items (rejects) cause increasing difficulties for 
the forwarding or return process. The goal is to resolve 
rejects whenever possible online, meaning while the mail is 

65 still located in the delay loop. If this should not be possible, 
it must be ensured that any related offline process is com- 
pleted before the mail leaves the discharge center. 
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For that reason, the code recognition system (OCR) is 
connected seamlessly with a 3-phase video coding, as shown 
in FIG. 5a. 

(1) If a clear OCR result does not exist, the missing 
information is supplemented in a first stage of the video 
coding through the input of a so-called accelerator keying 
sequence AKS 160. An AKS is a short, easily recogniz- 
able extract with fixed length, which resolves the outgo- 
ing and a portion of the incoming address information. If 
the AKS only leads to the conclusion 161 of a dictionary 
entry, then this result is validated with the OCR 162. With 
this, the location for delivering the mail is known and this 
result is sufficiently secured. If forwarding requests 163 
exist for this delivery location, an attempt to determine the 
name of the recipient 164 is made with the aid of the 
intermediate results from the OCR. If the name can be 
recognized 165, then a decision can be made as to whether 
the mail should be forwarded or returned. 

(2) If the AKS would permit several dictionary entries as 
result, an attempt is made to resolve this ambiguity with 
the aid of an OCR repeat attempt 166. If this attempt 
should prove successful 167, then the operation continues 
as described in the above under mail (1). 

(3) If a forwarding request exists in mail (1) for the delivery 
location to be considered, but the name of the recipient 
cannot be read 165 or 

if the OCR repeat attempt in mail (2) has not produced a 
definite result and one or several of the possible results, in 
this case meaning delivery locations, have a forwarding 
request 168: 

(4) -» then a priority for resolving this reject and thus also for 
resolving/processing a possible forwarding return 170 is 
transmitted to the online video coding location for selec- 
tions. 

(5) In the other case (if this is definitely not a forwarding 
candidate), the video coding order is started with a low 
priority in step 169. 

The phase 3 is explained in further detail with the aid of FIG. 
6. 

A decision 104, 161 must be made in this case whether a 
complete evaluation of the address information of a picture 
has occurred during the video coding. If the decision is 
positive (Yes), then, in case of forwarding instructions, the 
corresponding mail can be provided with a barcode follow- 
ing the reading of the recipient name and a comparison with 
the forwarding directory. The mail is then sorted further with 
standard means. In case of a negative decision (No), the 
invention provides for a further automatic evaluation by 
using the results of the video coding 166. It means that 
during this further automatic evaluation, the information 
gained through video coding along with the information 
shown on the image are available to the OCR processor. In 
the above example, this is the triple information "4432," 
"Hell," "8." In FIG. 6, this is expressed symbolically by the 
content of the circle 171. Following this, a decision 167 is 
made on whether a complete evaluation of the respective 
image has actually taken place. In the positive case (Yes), the 
respective information is used for a further sorting of the 
mail, in the same way as after a positive decision at decision 
point 104, 161. If the decision is negative (No), another 
video coding with high priority for forwarding requests and 
low priority without forwarding requests, the decision 168, 
takes place using the results of the further automatic evalu- 
ation. In that case, a number of alternatives for selection are 
presented to the operator, from which a selection must be 
made. 

If the new address cannot be determined in the time 
interval during which the respective mail is buffered in the 
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intermediate storage, then an identification (ID) code is 
printed onto the mail with the printer, so that the mail can 
continue to be processed offline. This occurs with high 
priority. 

The key factor with respect to time is that all items to be 
forwarded or returned are resolved before a mail item leaves 
the departure center. Other, simpler forms of video coding 
can be used for rejects that under no circumstances can be 
candidates for forwarding since a use of the transporting 
time to the video coding cannot be the cause of physical 
detours for these items. 

FIG. 7 shows a letter, which is to be forwarded following 
processing with the online system for the automatic pro- 
cessing of items to be forwarded and returned. The barcode 
affixed to the mail contains the new address of the recipient 
as target address. The forwarding instruction (e.g. 
"forward") and the new address, comprising street, house 
number and possibly apartment number, zip code and, if 
there is sufficient space, also the name of the location are 
printed to the right or left of the old address. 

This information is printed directly onto the mail either 
online in the OCR or in a sorting machine. It is printed on 
in bold, red type, which can still be read by the delivery 
person even if the background of the mail is restless. 

FIGS. 8 and 9 show a mail item to be returned following 
processing in the online system for automatic processing of 
items to be forwarded and returned. The barcode affixed to 
the mail contains the sender's address as the target address. 
The return notation (e.g. "return to sender") and the reason 
for the return (e.g. "address change") are printed on to the 
right or left of the old address. The sender address itself is 
not printed on again. It must be affixed to the mail by the 
sender him/herself in a form that can be read by humans if 
advance instructions are used. The information is printed 
directly onto the mail, online in the OCR. The print is bold 
and in red, making it possible for the delivery person to read 
it even if the mail background is restless. 

The items in FIGS. 8 and 9 differ in that on the mail in 
FIG. 9, the sender him herself printed on his/her customer 
number at the postal service and the type of advance 
instruction as code in the address field above the recipient 
address. 

The processing of such a return mail is considerably 
simpler than that of a mail item with "normal" advance 
instruction, as described in connection with FIG. 2: 
The search based on the ROI "sender address" (R01= 
regions of interest) and the reading of the sender 
address can be dropped. Instead, only the customer 
number of the sender is read as the top line in the 
address field. 

The parallel processing "detection of the advance instruc- 
tion" is dropped. The type of advance instruction is 
simply read with the code recognition system as the 
line below the top line in the address field. 

The return of a mail item prepared in this way requires 
much less time and resources than that of a normal 
mail, inscribed with advance instruction text and sender 
address. 

The preconditions for using a coded sender address and 
advance instruction are: 

1. An agreement between the postal service and the cus- 
tomer: The customer is accorded a customer number. 

2. A customer data bank at the postal service. 

3. A list with permissible codes for possible advance instruc- 
tions. 

4. The coding of customer number and advance instruction 
in the address field, above the recipient's address. 
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5. The affixing of a sender address that can be read by the 
delivery person, so that the delivery person can deliver the 
return. 

The FIGS. 10, 11 and 12 demonstrate the treatment of a 
return based on a delivery instruction. 5 

FIG. 10 shows a letter, which was not recognized in the 
online system for the automatic processing of items to be 
forwarded or returned as a candidate for forwarding or 
return because no information on an address change was 
available to the system. The barcode for the address, rec- 10 
ognized as correct, was therefore printed onto the letter and 
it was sent to the delivery person responsible for the former 
residence of the recipient. The delivery person only can 
determine that: 

the recipient refuses to accept; 15 
has moved, address unknown; 
has died or is resolved. 

As shown in FIG. 11, the delivery person therefore stamps 
the letter as return (affixing a delivery notation) and manu- 
ally marks the reason for the return. An empty label is pasted 20 
over the old barcode, which is thus made unrecognizable. In 
this way, the letter can be processed further by machine 
instead of by hand. The delivery person will send it directly 
to the office of origin sorting. 

FIG. 12 shows the same letter following processing by the 25 
online system for the automatic processing of items to be 
forwarded and returned. The detection of the delivery 
instruction has recognized the return stamp from the deliv- 
ery person and has sent the mail to the return processing (as 
described in FIGS. 2, 3 and 4). There, the barcode for the 
sender address and the return instruction "return to sender" 
(optional, in bold red text) are affixed online in the OCR onto 
the empty label. 

FIG. 13 illustrates the machine configuration for the 
online system for the automatic processing of items to be 
forwarded or returned. A mail item 400 is fed into an 
integrated address reading and video coding system, here 
referred to as "presorting " In this machine, it is scanned 
with a highly developed scanner 402 after it has passed by 
the pre-barcode detector 401. The scanned pixel image is 
sent to the recognition system 404, for which the task and 
function were described in detail in the preceding explana- 
tions. The recognition system is supported by a video coding 
system where video-coding personnel at video coding sta- 
tions 409 either input the address information or make 
selections from lists with alternatives. A video control com- 
puter 418 controls the transmission of images from the 
recognition system to the video coding system, as well as the 
return transmission of the coded results to the machine 
control 410, to permit the imprinting of the items with the 
target barcode 403, as well as the sorting into the correct 
rough sorting compartment 407. 

Items which cannot be evaluated online during the inter- 
mediate storage will be imprinted with an ID-tag (with the 
barcode printer 403), which is stored in the ID-tag server 
419, together with the sorting code information, the com- 
plete address block information and the information on 
possibly existing delivery notations or advance instructions. 

The rough sorting compartments 407 contain items with 
correct addresses as well as items that must be forwarded or 60 
returned, for which the addresses could be corrected with the 
online system for the automatic processing of items to be 
forwarded and returned. This flow of letters moves directly 
into the next processing stage, which is generally the final 
sorting stage. 65 

In the detailed final sorting machine, here referred to as 
"final sorting," the barcode of each mail is read with a 
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barcode reader 422. Once a target code is recognized, the 
mail is moved directly to the associated final sorting com- 
partment 420. If an ID-tag is recognized, then an ID-tag 
request is sent to the ID-tag server 419. Following such a 
request, the server transmits the necessary sorting, address 
and other information, stored under this ID, to the control of 
the final sorting machine 421. The mail to be forwarded is 
imprinted with the notation "forward" or "back to sender," 
the new address or the reasons for the return. This is done 
with the aid of a text printer 423 in the detailed final sorting 
machine. Subsequently, the mail is selected and moved 
according to the target information to the corresponding 
final sorting compartment 420. 

A small number of items that could not be identified 
online are moved to a special rough sorting compartment 
406 during the presorting. These items are marked with an 
ID-tag. The pixel images of these items are stored in an 
image handler 411 and are subsequently sent with highest 
priority to the video coding. The results from the video 
coding are stored under the ID -tag in the ID-tag server. The 
items are then again moved to the presorting, which selects 
them to be moved to the respective rough-sorting compart- 
ment following an ID-tag query to the ID-tag server. Fol- 
lowing that, they can be moved with the normal flow of 
letters to the final sorting stage. 

To be sure, items that carry a delivery notation (identified 
by the delivery person as return items and marked with a 
"return to sender" stamp), but which have not yet been given 
a barcode, cannot be processed further. They must be moved 
to a special Rough sorting compartment 405. 

What is claimed is: 

1. A method for online processing of items of mail to be 
forwarded, including the following online processing steps: 

a. making an image of a side of respective mail items that 
contain information including names and addresses, 
and transporting the items to an intermediate storage; 

b. digitizing and storing the respective images, 

c. determining areas containing interesting information, 
including sender name and sender address, receiver 
name and receiver address, and instructions for for- 
warding or returning the respective mail items; 

d. classifying areas containing interesting information, 

e. automatically reading in of instructions for returning or 
forwarding of mail items and reading of recipient name 
and address by means of an OCR reader, including 
automatically checking of results in a name address 
data bank and, in case they do not coincide, in another 
data bank where forwarding requests relative to deliv- 
ery points are stored, wherein only the sender address 
is read in case of determination of a return notation 
made by a delivery person on the mail item; 

f. online evaluating recipient names and recipient 
addresses that are not clearly read by the OCR reader, 
including checking the name/address data bank, and the 
data banks for forwarding with the aid of video coding 
of the images of each item of mail for which at least one 
forwarding instruction exists for possible deliver 
points, by taking into account the forwarding or return 
instructions; 

g. printing, legibly and encoded, a new recipient address, 
and forwarding instructions on the mail items leaving 
the intermediate storage; and 

h. distributing the mail items according to the new address 
and the forwarding instructions. 

2. The method according to claim 1, wherein coding 
according to predetermined rules takes place during the first 
video coding. 



09/30/2003, EAST Version: 1.04.0000 



US 6,292, 

11 

3. The method according to claim 1, wherein a selection 
coding takes place during the additional video coding, in 
such a way that a selection is made from unclear results of 
the coding. 

4. The method according to claim 1, further including s 
supplying a return instruction to the video coding in case 
return instructions from a postal carrier cannot be read with 
certainty by means of an OCR reader. 

5. The method according to claim 1, including supplying 

a sender address to the video coding if it cannot be read 10 
automatically and with certainty by means of an OCR 
reader. 

6. The method according to claim 1, including imprinting 
advance instructions by the sender, and one of the sender 
address and customer number onto a recipient address field 15 
in addition to the legible information. 

7. A device for carrying out the method according to claim 
1, comprising: 

a device for generating and storing images of items; 

20 

an OCR reader for automatic evaluation of information on 
the items, the automatic evaluation including a result of 
checks in a name/address data bank and a forwarding 
directory; 

a device for video coding the mail images, said device J5 
having a plurality of video coding stations, including 
means for checking the results of a comparison of the 
image information in the name/address data bank and 
the forwarding directory; 

an image controller for controlling the flow of data 30 
between the device for the automatic evaluation and the 
device for video coding, the image controller 
supplying, during the video coding, images of the items 
not identified with certainty to the OCR reader for 
further automatic evaluation by using the results of the 35 
video coding, wherein the images of items for which 
possible delivery points for at least one forwarding 
instruction exist are supplied first to the device for 
video coding; 

a printing device for printing on addresses in legible and 40 
encoded forms, and for printing on forwarding instruc- 
tions; 

an intermediate storage; and 

at least one sorting device. ^ 

8. A method according to claim 1, comprising the follow- 
ing processing steps for realizing the video encoding of the 
processing step f: 

transmitting the images of the items, not read with cer- 
tainty in the automatic OCR evaluation, to a first video 5Q 
coding; 

feeding of those images, which were not evaluated with 
certainty during the video coding, to another automatic 
evaluation in the OCR reader by taking into accounting 
the results of the first video coding; and 
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evaluating images of items not evaluated with certainty 
during the further automatic evaluation, said items 
containing at least one forwarding instruction for pos- 
sible delivery points, during another video coding by 
using the results from the additional automatic evalu- 
ation. 

9. A method for online processing of items of mail to be 
forwarded, including the following processing steps: 

making an image of a side of respective mail items that 
contain information including names and addresses, 
and transporting the items to an intermediate storage; 

digitizing and storing the respective images, 

determining areas containing interesting information, 
including sender name and sender address, receiver 
name and receiver address, and instructions for for- 
warding or returning respective mail items; 

classifying areas containing interesting information; 

automatically reading in of instructions for returning or 
forwarding of mail items and reading of recipient name 
and address by means of an OCR reader, including 
automatically checking of results in a name/address 
data bank and, in case they do not coincide, in another 
data bank where forwarding requests relative to deliv- 
ery points are stored, wherein only the sender address 
is read in case of a determination of a return notation 
made by a delivery person on the mail item; 

evaluating recipient names and recipient addresses that 
are not clearly read by the OCR reader, including 
checking the name/address data bank, and the data 
banks for forwarding with the aid of video coding of the 
images of each item of mail for which at least one 
forwarding instruction exists for possible deliver 
points, by taking into account the forwarding or return 
instructions; said evaluating step including: 

transmitting the images of the items, not read with cer- 
tainty in the automatic OCR evaluation, to a first video 
coding; 

feeding of those images, which were not evaluated with 
certainty during the video coding, to another automatic 
evaluation in the OCR reader by taking into accounting 
the results of the first video coding; and 

evaluating images of items not evaluated with certainty 
during the further automatic evaluation, said items 
containing at least one forwarding instruction for pos- 
sible delivery points, during another video coding by 
using the results from the additional automatic evalu- 
ation; 

printing, legibly and encoded, a new recipient address, 
and forwarding instructions on the mail items leaving 
the intermediate storage; and 

distributing the mail items according to the new address 
and the following instructions. 

***** 
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